Local-aware spatio-temporal attention network with multi-stage feature fusion for human action recognition

Abstract In the study of human action recognition, two-stream networks have made excellent progress recently. However, there remain challenges in distinguishing similar actions videos. This paper proposes a novel local-aware spatio-temporal attention network with multi-stage feature fusion based on compact bilinear pooling for recognition. To elaborate, taking as our essential backbones, spatial first employs multiple transformer parallel manner to locate discriminative regions related actions. Then, we perform between local and global features enhance representation. Furthermore, output temporal information are fused at particular layer learn pixel-wise correspondences. After that, bring together three outputs generate descriptors verify efficacy proposed approach, comparison experiments conducted traditional hand-engineered IDT algorithms, classical machine learning methods (i.e., SVM) state-of-the-art deep multiplier networks). According results, approach is reported obtain best performance among existing works, accuracy 95.3% 72.9% UCF101 HMDB51, respectively. The experimental results thus demonstrate superiority significance architecture solving task